1. Field of the Invention
The present invention relates to multispectral, hyperspectral, and ultraspectral imagery, and more particularly to formation of spatially correct spectral two dimensional spectral images from stored digital single dimensional images by digital manipulation of stored data. In the preferred embodiment, single dimensional images of a selected spectral band are correlated to their respective geometric locations by referring to linked two dimensional images having a plurality of geometric reference points, and are subsequently assembled into a single, two dimensional image of the selected spectral band.
2. Description of the Prior Art
Aerial imagery provides a specialized technique of obtaining information regarding characteristics of large land masses which may be unavailable from other sources or inefficiently obtained in other ways. In some image acquisition processes, images are arranged or taken by layers of information. Such images have equivalent geographic coverage, but capture different characteristics. For example, multispectral or hyperspectral imagery includes layers of data taken at different frequency bands. Each image data layer conveys different information to the observer. In the example of forest fires, images derived only from visible portions of the spectrum may reveal only a cloudy smoke cover. However, images based on infrared bands may reveal underlying conditions including flame front lines. In agriculture, to consider a second example, health of a crop, as reflected by sparseness or thickness of foliage, may be quantified to determine where certain nutrients or water are deficient or overabundant.
Successful sequential imagery, particularly wherein bounds of one image abut bounds of surrounding images, is dependent upon appropriate attitude of the image acquiring device. Low altitude multispectral, hyperspectral, and ultraspectral imagery, that being imagery acquired from a camera or other image gathering apparatus carried aboard an air breathing aircraft, is especially subject to various problems which may arise from attitude of the aircraft. A notorious obstacle to taking sequential single dimensional or line images from aircraft is variation in any of pitch, roll, and yaw of the aircraft due to air currents, internal vibration, and other causes. As pitch, roll, and yaw vary, each succeeding imaging location varies from a desired constant orientation, such as directly below the aircraft. Consequently, captured images may not be sufficiently aligned with previously captured images as to display sufficient continuity of the subject to render useful information. In xe2x80x9cpush broomxe2x80x9d multispectral, hyperspectral, and ultraspectral scanning, the location of images varies from the usual focal point directly below the aircraft with aircraft movement.
In aerial photography, a frequent answer has been to stabilize a camera within the aircraft with respect to the ground so that the camera maintains vertical orientation despite variations in pitch, roll, and yaw. This approach has been satisfactory when the aircraft is fairly large. That is to say, conventional manned aircraft usually have sufficient space, strength, and power to accommodate stabilizing apparatus. U.S. Pat. No. 4,764,781, issued to Kenneth G. Leib et al. on Aug. 16, 1988, exemplifies camera stabilization.
However, stabilizing apparatus precludes the use of miniature aircraft, where xe2x80x9cminiaturexe2x80x9d encompasses aircraft too small to accommodate a human pilot. In some situations, only miniature aircraft may be practical to use. Even where larger aircraft are employed, it may be preferable to conserve space, weight, and expense, among other resources. A need exists for much smaller apparatus that will solve the problem of image alignment.
Capture of images in digital form and manipulation of digital data may be exploited to render stabilizing apparatus unnecessary. Digital apparatus is sufficiently small as to reduce requirements for space, weight, complexity and reliability, and expense in aerial imagery acquisition. Correlation techniques applicable to digitized images are known. However, many correlation techniques are not susceptible to single dimensional images typical of images acquired by xe2x80x9cpush broomxe2x80x9d hyperspectral scanners because it is necessary to have a minimum of three non-collinear points to establish reference data from two dimensional images. No satisfactory way to manipulate single dimensional images exists to establish accurate location in two dimensions.
One approach is to generate several images simultaneously from a common source. This approach is employed in holography and other optical disciplines. U.S. Pat. No. 3,586,434, issued to Peter F. Mueller on Jun. 22, 1971, shows use of a beam splitter in multispectral image storage.
U.S. Pat. No. 5,619,596, issued to Tadao Iwaki et al. on Apr. 8, 1997 shows a correlation technique that overcomes motion of an input image.
U.S. Pat. No. 5,327,286, issued to Jeffrey B. Sampsell et al. on Jul. 5, 1994, describes an optical correlation system employing a beam splitter to derive two images, and a microprocessor to process corresponding data.
U.S. Pat. No. 5,061,063, issued to David P. Casasent on Oct. 29, 1991, presents a technique wherein a single dimension image is compared to a two dimensional image.
U.S. Pat. No. 6,160,910, issued to Daniel Freifeld on Dec. 12, 2000, shows an optical system for providing images of sufficient quality to derive accurate measurements from the same.
The prior art fails to show a system wherein a single dimensional image and a two dimensional image are generated simultaneously from a single optical input, are digitized, and wherein a two dimensional image is reconstituted from a digital record of many single dimensional images, using geometric reference points taken from the two dimensional image to establish correct location of each single dimensional image within the reconstituted image.
None of the above inventions and patents, taken either singly or in combination, is seen to describe the instant invention as claimed.
The present invention sets forth novel apparatus for and method of registering a single dimensional image or simultaneously collected series of spectrally unique single dimensional images to a two dimensional image, and of generating two dimensional images, preferably of a selected spectral band, from stored digital single dimensional images by digital manipulation of stored data. Single dimensional images of a selected spectral band are correlated to their respective geographic locations by referring to linked two dimensional images having a plurality of geographic reference points, and are assembled into a single, two dimensional image of the selected spectral band.
The foregoing is accomplished by processing optical images to generate two images simultaneously. A light beam conveying a desired image, which will be termed a primary image, is split into two images. One of these two images is passed through a slit to generate a single dimensional image, and preferably is also divided into spectral bands, which process yields a number of single dimensional images differing only in their spectral bands. The other image, which will be termed a reference image, is retained in two dimensional format, and includes a plurality of at least three recognizable geographic reference points. All images, wherein each primary image yields one reference image and potentially many single dimensional images or line scans each of one spectral band, are stored digitally, and are digitally linked to one another. This process is repeated for each sequential primary image.
A two dimensional image of a selected spectral band is subsequently reconstituted from the many single dimensional images which have been digitally stored. Each single dimensional image or line is placed digitally in an array corresponding to the primary image. Each line is correctly located by referring to its linked reference image. The reference image includes at least three recognizable geometric reference points. Because location of each line scan can be related to reference points also employed for other line scans taken from other primary images, it follows that all line scans can be appropriately arrayed within a reconstituted spectral two dimensional image by relating each constituent line scan to a common set of reference points. This is done by applying appropriate well known mathematical manipulation techniques to both the reference images and to each line scan.
A principal application of the invention is that of aerial photography. Heavy, complex, bulky, expensive stabilization equipment can be eliminated in favor of lighter, less complicated, compact, less expensive optical and digital equipment. In aerial photography, it is contemplated that the present invention can in many applications so reduce imagery apparatus in dimensions and weight that miniature, unmanned aircraft can be employed in place of large, manned aircraft. This has obvious implications relating to safety of personnel both on the aircraft and also on the ground.
Although description of the invention is primarily couched in terms of low altitude aerial photography, it must be recognized that the novel principles apply to other forms of image acquisition. For example, satellite imagery and other high altitude aerial imagery, underwater imagery, robotic and other imagery conducted by machines, and micro-imagery, to name only a few possible other fields of imagery, may also utilize the invention. The present invention applies to all fields wherein single dimensional images or line scans are obtained from two dimensional primary image acquisition, are stored, and are subsequently reconstituted by registering each line scan with a linked two dimensional image containing at least three geometric points of reference.
Elements of apparatus for carrying out the invention include a camera type aperture, a beam splitter, a first digital recorder for recording two dimensional reference images, a beam restrictor such as a slit disposed to intercept the second beam, for generating single dimensional images or line scans, an optional diffractor for generating multispectral single dimensional images, and a second digital recorder for recording the second images. The two digital recorders are, of course, operably linked to memory devices. Desired images may be subsequently reconstituted from the stored records by a suitable computer or comparable digital processor into which the digital record has been entered, using known mathematical techniques to manipulate the data.
In summary, the invention provides a way of generating two dimensional images from line scans preferably taken sequentially from a mobile platform passing over an area of interest. Conventional stabilization apparatus for cameras and other image acquisition apparatus is eliminated in favor of data manipulation wherein a reference image is correlated with a subject image in a manner eliminating problems of parallax. The images are stored in manipulatable and retrievable digital form.
Accordingly, it is one object of the invention to register a single dimensional image to a two dimensional reference image by digital manipulation.
Another object of the invention is to generate two dimensional images from line scans taken sequentially from a mobile platform passing over an area of interest in a manner overcoming shortcomings of the prior art relating to stabilization of image acquisition apparatus.
Still another object of the invention to reduce bulk, complexity, weight, and cost of apparatus the function of which is to assure that subject images can be accurately correlated to reference images in a manner eliminating problems such as parallax.
It is a further object of the invention to record single dimensional subject images and two dimensional reference images, and to reconstitute two dimensional subject images digitally from the recorded images.
It is an object of the invention to provide improved elements and arrangements thereof in an apparatus for the purposes described which is inexpensive, dependable and fully effective in accomplishing its intended purposes.
These and other objects of the present invention will become readily apparent upon further review of the following specification and drawings.